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April  7, 1999 

MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  (COMPTROLLER) 

UNDER  SECRETARY  OF  DEFENSE  (PERSONNEL  AND 
READINESS) 

SUBJECT:  Audit  Report  on  Data  Su^rting  the  FY  1998  DoD  Military  Retirement 
Health  Benefits  Liability  Intimate  (Report  No.  99-127) 


We  are  providing  this  report  for  information  and  use.  We  conducted  the  audit 
in  response  to  die  Chief  Financial  Officers  Act  of  1990  and  the  Federal  Financial 
Management  Act  of  1994.  We  considered  management  comments  on  a  draft  of  this 
report  in  preparing  die  final  report. 

Comments  received  on  a  draft  of  this  report  conformed  to  the  requirements  of 
DoD  Directive  7650.3  and  left  no  unresolved  issues.  Therefore,  no  additional 
comments  are  required. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Mr.  Charles  J.  ^chardson,  at  (703)  604-958^2  (DSN  664-9582) 
(crichardson@dodig.osd.mil)  or  Mr.  Walter  R.  Loder,  at  (703)  604-9534 
(DSN  664-9534)  (wrloder@dodig.osd.mil).  See  Appendix  B  for  the  report  distribution. 
The  audit  team  mem^rs  are  listed  inside  the  back  cover. 

David  K.  Steensma 
Deputy  Assistant  Injector  General 
for  Auditing 
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Executive  Summary 


Introduction.  The  audit  (one  in  a  series  of  audits  to  review  the  reliability  of  data 
elements  used  in  the  estimate  of  the  Military  Retirement  Health  Benefits  Liability)  was 
performed  in  support  of  the  Chief  Financial  Officers  Act  of  1990  (Public  Law  101-576) 
and  the  Federal  Financial  Management  Act  of  1994  (Public  Law  103-356).  Public 
Law  103-356  requires  DoD  and  other  Government  agencies  to  prepare  financial 
statements  for  FY  1996  and  each  succeeding  year.  The  DoD  FY  1998  financial 
statements  reported  a  $223  billion  unfimded  liability  for  DoD  military  retirement  health 
benefits  for  FY  1998.  The  FY  1998  liability  was  based  on  the  FY  1997  $218  billion 
estimate,  projected  to  September  30,  1998.  In  addition,  the  $223  billion  unfunded 
liability  was  24  percent  of  the  $948.5  billion  of  liabilities  included  on  the  DoD-wide 
financial  statements  and  8  percent  of  the  estimated  $2.7  trillion  of  the  Federal 
Employee  and  Veteran  Benefits  payable.  Military  retirement  health  benefits  are 
post-retirement  benefits  that  DoD  provides  to  military  retirees  and  other  eligible 
beneficiaries  through  the  Civilian  Health  and  Medical  Program  of  the  Uniformed 
Services  and  DoD  militaty  treatment  facilities.  Eligible  beneficiaries  also  may  obtain 
medical  care  from  the  private  sector  under  the  Civilian  Health  and  Medical  Program  of 
the  Uniformed  Services.  The  Civilian  Health  and  Medical  Program  of  the  Uniformed 
Services  accounts  for  $51  billion  of  the  $223  billion  unfunded  military  retirement  health 
benefits  liability  and  the  remaining  $172  billion  of  the  liability  is  for  medical  care  that 
the  DoD  military  treatment  facilities  will  provide  to  eligible  beneficiaries. 

Objectives.  The  overall  audit  objective  was  to  assess  the  reliability  and  completeness 
of  Ae  data  used  to  calculate  the  DoD  Military  Retirement  Health  Benefits  Liability  for 
FY  1998.  Specifically,  we  reviewed  the  ouq)atient  visit  data  contained  in  the 
Composite  Health  Care  System  for  reliability  and  completeness.  Additionally,  we 
reviewed  the  management  controls  as  they  related  to  the  objective. 

Results.  The  quality  of  the  Composite  Health  Care  System  outpatient  visit  data  was 
unreliable  for  use  in  developing  the  military  retirement  health  benefits  liability  estimate. 
As  a  result,  we  were  unable  to  substantiate  the  validity  and  completeness  of  the 
Composite  Health  Care  System  ouq)atient  visit  data  that  the  Office  of  the  Actuary, 

DoD,  will  use  to  support  the  calculation  of  the  FY  1998  actuarial  estimate  of  the 
military  retirement  health  benefits  liability.  Outpatient  medical  care  services  at  DoD 
military  treatment  facilities  represented  more  than  $100  billion  of  the  $223  billion 
un^nded  military  retirement  health  benefits  liability  for  FY  1998.  Therefore,  the 
failure  to  ensure  the  accuracy  of  the  outpatient  data  that  will  be  used  to  calculate  an 
estimate  of  the  unfunded  liability  for  FY  1998  constitutes  a  significant  impediment  to 
achieving  unqualified  audit  opinions  on  both  the  DoD  and  Government-wide  financial 
statements  for  FY  1999  and  future  years.  The  Office  of  the  Assistant  Secretary  of 
Defense  (Health  Affairs)  is  initiating  actions  to  address  the  issues  identified  in  this  audit 
report.  See  the  Finding  for  details  of  the  audit  results. 
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The  lack  of  controls  over  the  Composite  Health  Care  System  outpatient  data  was  a 
material  control  management  control  weakness.  See  Appendix  A  for  details. 

Summary  of  Recommendations.  We  recommend  that  the  Assistant  Secretary  of 
Defense  (Health  Affairs)  develop  and  implement  a  data  quality  assurance  program  for 
recording  and  reporting  the  Composite  Health  Care  System  outpatient  workload  and 
related  documentation,  develop  and  implement  standardized  appointment  data  elements 
for  similar  me(Kcal  care  in  like  clinics,  and  conduct  follow-up  tests  to  determine  if  the 
quality  controls  and  standardized  appointment  data  elements  have  improved. 

Management  Comments.  The  Assistant  Secretary  of  Defense  (health  Affairs) 
concurred  with  the  recommendations,  stating  that  increased  quality  assurance  and 
management  control  programs  were  being  developed  for  all  the  Military  Health  System 
information  systems.  In  addition,  a  Tri-Service  group  was  established  to  develop  DoD 
standardized  appointment  types,  and  ongoing  management  review  and  monitoring  of 
quality  controls  will  be  incorporated  in  the  design  of  the  data  quality  assurance 
program.  However,  the  Assistant  Secretary  did  not  fully  concur  with  the  finding, 
stating  fiiat  the  Composite  Health  Care  System  outpatient  visit  data  is  reasonably 
reliable  for  use  in  developing  the  military  retirement  health  benefits  liability  estimate. 
See  the  Finding  for  the  complete  discussion  of  management  comments  and  Management 
Comments  for  the  complete  text  of  the  comments. 

The  Air  Force  agreed  also  responded  and  agreed  with  the  recommendations,  stating  that 
the  new  Air  Force  Instruction  41-210,  Patient  Administration  Functions,  contains 
management  control  guidance  to  help  establish  a  consistent  approach  to  ensuring  the 
reliability  of  outpatient  data.  In  addition,  the  new  Air  Force  Instruction  41-210 
addresses  a  data  quality  assurance  program  implementation  at  each  Air  Force  military 
treatment  facility  and  reinforces  the  importance  and  requirement  to  audit  outpatient  visit 
data  to  ensure  reliability. 

Audit  Response.  The  Assistant  Secretary’s  comments  to  the  recommendations  were 
responsive.  However,  we  do  not  agree  with  the  Assistant  Secretary’s  comments  that 
the  outpatient  visit  data  is  reasonably  reliable.  The  Assistant  Secret^’s  comment  that 
there  was  a  need  for  data  quality  controls  and  that  there  was  a  material  ma^gement 
control  wea^ess  contradicts  the  reasonably  reliable  statement.  We  appreciate  the 
cooperative  efforts  of  the  Assistant  Secretary  to  improve  the  accuracy  and  quality  of  tiie 
information  from  all  of  the  information  system. 
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Background 


Requirements  for  Financial  Statements.  Public  Law  101-576,  “The  Chief 
Financial  Officers  Act  of  1990,”  as  amended  by  Public  Law  103-356,  “The 
Federal  Financial  Management  Act  of  1994,”  requires  DoD  and  other 
Government  agencies  to  prepare  consolidated  financial  statements  for  FY  1996 
and  each  succeeding  year. 

Requirements  for  Financial  Statement  Audits.  Under  the  Chief  Financial 
Officers  Act  of  1990,  die  Inspector  General,  DoD,  is  responsible  for  the  audit  of 
the  financial  statements  of  DoD.  The  General  Accounting  Office  is  responsible 
for  the  audit  of  the  consolidated  financial  statements  of  the  U.S.  Government. 

Office  of  Management  and  Budget  Form  and  Content  Guidance.  Office  of 
Management  and  Budget  Bulletin  No.  97-01,  “Form  and  Content  of  Agency 
Financial  Statements,”  October  16,  1996,. provides  general  and  specific 
guidance  to  agencies  of  the  Government  on  the  preparation  of  financial 
statements.  Office  of  Management  and  Budget  Bulletin  No.  97-01  established 
gui^ce  for  reporting  pensions,  other  retirement  benefits,  and  other 
post-employment  benefits.  l^D  other  retirement  benefits  include  all  retirement 
benefits  other  than  pension  plan  benefits.  Office  of  Management  and  Budget 
Bulletin  No.  97-01  further  states  that  entities  that  are  responsible  for  accounting 
for  “other  retirement  benefits”  should  calculate  and  report  the  liabilities  and 
related  expenses  in  accordance  with  the  Statement  of  Federal  Financial 
Accounting  Standard  No.  5,  “Accounting  for  Liabilities  of  the  Federal 
Government,”  September  1995. 

Materiality  of  Liability.  FY  1997  was  the  first  year  that  DoD  reported  the 
unfunded  liability  for  the  DoD  military  retirees  medical  health  benefite  on  the 
DoD-wide  consolidated  financial  statements.  DoD  reported  a  $218  billion 
unfunded  liability  for  DoD  military  retirement  health  benefits  for  FY  1997  and  a 
$223  billion  un^nded  liability  for  FY  1998.  The  FY  1998  $223  billion 
unfunded  liability  was  24  percent  of  the  $948.5  billion  of  liabilities  included  on 
the  DoD-wide  financial  statements  and  8  percent  of  the  estimated  $2.7  trillion  of 
the  Federal  Employee  and  Veteran  Benefits  Payable.  The  Civilian  Health  and 
Medical  Program  of  the  Uniformed  Services  accounts  for  $51  billion  of  the 
$223  billion  unfunded  military  retirement  health  benefits  liability  and  the 
remaining  $172  billion  of  the  liability  is  for  medical  care  that  the  DoD  military 
treatment  facilities  will  provide  to  eligible  beneficiaries.  Therefore,  the  DoD 
military  retirement  medical  health  benefits  liability  is  material  to  the  DoD-wide 
financial  statements  and  Federal  Government  financial  statements. 

Armed  Forces  Medical  Care.  United  States  Code  Annotated,  Title  10  Armed 
Forces,  “Chapter  55-Medical  and  Dental  Care,”  requires  DoD  to  provide  a 
uniform  program  of  medical  and  dental  care  for  uniform  service  members, 
certain  former  members  of  those  services,  and  for  dependents. 

Military  Treatment  Facilities.  Approximately  $171  billion  of  the  $223  billion 
FY  1998  estimated  military  retirement  health  benefits  liability  represented  foture 
outpatient  and  inpatient  medical  care  that  the  DoD  milita^  treatment  facilities 
are  expected  to  provide  to  eligible  beneficiaries.  DoD  military  treatment 
facilities  include  hospitals  and  clinics.  DoD  has  104  military  treatment  facilities 
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designated  as  host  sites  for  the  Composite  Health  Care  System  (CHCS).  A  host 
site  is  a  PoD  military  treatment  facility  with  a  CHCS  server.  The  remaining 
$51  billion  of  the  $223  billion  liability  will  be  provided  through  the  TRICAfe 
program.  The  TRICARE  program  provides  health  care  through  civilian  medical 
providers. 

CHCS  Outpatient  Data  Supports  Liability  Calculations.  The  Office  of  the 
Actuary,  DoD,  relies  on  outpatient  visit  data  that  the  DoD  milimry  treatment 
facilities  record  within  the  CHCS,  to  calculate  the  PoD  military  treatment 
focilities  portion  of  the  estimated  liability.  The  CHCS  is  a  comprehensive 
medical  information  system  that  DoD  devel^d  to  provide  automated  support  to 
its  military  treatment  facilities.  The  CHCS  is  composed  of  integrated  modules 
that  facilitate  the  collection  and  input  of  data  at  the  point  of  medical  care. 

The  Office  of  the  Actuary,  DoD,  developed  a  methodology  that  uses  CHCS 
outpatient  visit  data  to  calculate  an  estimated  average  cost  to  provide  ou^atient 
care  in  DoD  milita^  treatment.facilities  to  eligible  beneficiaries  by  selective  age 
categories.  Qu^atient  medical  care  includes  visits  by  an  authorized  patient  to  a 
separate,  organized  clinic  or  specialty  service  for  examination,  diagnosis, 
treatment,  evaluation,  consultation,  counseling,  and  medical  advice. 

Contract  and  Actuarial  Tedmical  Support.  The  Office  of  the  Actuary,  DoD, 
is  responsible  for  tiie  valuation  of  the  military  retirement  health  benefits 
liability.  Milliman  and  Robertson,  Inc.,  an  actuary  and  consultant  firm,  was 
contracted  to  assist  the  Office  of  the  Actuary,  DoD,  in  calculating  an  estimate  of 
the  unfunded  liability  for  the  military  post-retirement  health  care  benefits 
through  FY  2000. 

The  contractor  calculated  the  $218  billion  estimate  for  the  FY  1997  military 
retirement  health  benefits  liability  based  on  the  same  data  used  to  compute  the 
initial  FY  1997  $210  billion  liability  estimate.  The  $210  billion  liability 
increased  for  the  following  reasons: 

•  The  claims  cost  data  used  to  develop  the  $218  billion  estimate  were 
projected  an  extra  year  to  October  1,  1997,  using  the  medical  trend 
assumption  of  2.5  percent  for  FY  94  through  95  from  the  FY  1997 
report. 

•  The  $218  billion  estimate  was  based  on  a  method  of  spreading  cost 
over  years  of  service  rather  than  as  a  level  percentage  of  payrolls. 
Spreading  cost  over  years  of  service  is  the  method  specified  in 
Statement  of  Federal  Financial  Accounting  Standard  No.  5. 
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Objectives 

The  overall  objective  was  to  assess  the  reliability  and  completeness  of  the  data 
used  to  calculate  the  DoD  Military  Retirement  Health  Benefits  Liability  for 
FY  1998.  Specifically,  we  reviewed  the  outpatient  visit  data  contained  in  the 
Composite  Health  Care  System  for  reliability  and  completeness.  Additionally, 
we  reviewed  the  management  controls  as  related  to  the  objective.  See 
Appendix  A  for  a  discussion  of  tiie  audit  scope  and  methodology,  our  review  of 
the  management  control  program,  and  a  summary  of  prior  audit  coverage 
related  to  the  audit  objectives. 
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Management  Controls  Over  Military 
Treatment  Facilities  Outpatient 
Visit  Data 

The  quality  of  the  CHCS  outpatient  visit  data  was  unreliable  for  use  in 
(teveloping  the  military  retirement  health  benefits  liability  estimate.  The 
CHCS  outpatient  visit  data  was  unreliable  because  the  Office  of  the 
Assistant  Secretary  of  Defense  (Health  Affairs)  did  not  develop  and 
implement  management  controls  in  the  form  of  a  quality  assurance 
program  for  CHCS  outpatient  visit  data.  Further,  stan^rdized 
appointment  data  elements  for  similar  clinics  were  not  developed  for 
recording  outpatient  visit  data  in  the  CHCS.  As  a  result,  we  were  unable 
to  substantiate  the  validity  and  completeness  of  the  CHCS  ouq[)atient  visit 
data  that  the  Office  of  the  Actuary,  DoD,  will  use  to  support  the 
calculation  of  the  FY  1998  actuarial  estimate  of  the  miiita^  retirement 
health  benefits  liability.  The  DoD  military  treatment  facilities  outpatient 
medical  care  services  represented  more  than  $100  billion  of  the 
$223  billion  FY  1998  military  retirement  health  benetits  liability.  The 
inability  to  ensure  the  accuracy  of  the  CHCS  outpatient  visit  data 
constitutes  a  significant  impediment  to  achieving  unqualified  audit 
opinions  on  both  the  DoD  and  Oovemment-wide  financial  statements  for 
In  1999  and  future  years.  The  Office  of  the  Assistant  Secretary  of 
Defense  (Health  Affairs)  is  initiating  actions  to  address  the  issues 
identitied  in  this  audit  report. 

Guidance  for  Counting  Outpatient  Visits 

Outpatient  visits  are  countable  visits  made  by  authorized  personnel  to  medical 
clinics.  DoD  6010. 13-M^  Medical  Expense  and  Performance  Reporting  System 
for  Fixed  Military  Medical  and  Dental  Treatment  Facilities,  October  1995,  and 
DoD  6010. 13-M,  Change  1,  July  9,  1997,  established  guidmce  for  determining 
countable  visits.  This  guidance  requires  that  the  DoD  military  treatment 
facilities  clinics  satisfy  three  criteria  before  counting  a  visit  as  a  valid  outpatient 
visit: 


•  There  must  be  interaction  between  an  authorized  patient  and  a 
healthcare  provider. 

•  Independent  judgment  about  the  patient’s  care  must  be  used, 
assessment  of  the  patient’s  condition  must  be  made,  and  any  one  or 
more  of  the  following  must  be  accomplished: 

•  examination, 

•  diagnosis, 

•  counseling,  or 


•  treatment. 
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•  The  DoD  military  treatment  facilities  must  include  adequate 
documentation  in  the  patient’s  authorized  record  of  medical  treatment. 
At  a  minimum,  the  documentation  must  include  the  date  of  the  visit, 
name  of  clinic,  reason  for  visit,  assessment  of  the  patient,  description 
of  the  interaction,  disposition,  and  the  signahire  of  the  healthcare 
provider.  In  all  instances,  the  DoD  military  treatment  facilities  must 
maintain  a  clear  and  acceptable  audit  trail. 

In  addition,  telephone  consultations  that  meet  the  DoD  criteria  are  also  counted 
as  visits. 

Quality  of  CHCS  Outpatient  Workload  Data  is  Unreliable 

We  reviewed  998  CHCS  outpatient  visits  and  determined  that  606  of  these  did 
not  satisfy  the  DoD  criteria  for  determining  countable  visits  (see  Table  1).  For 
example,  506  of  the  606  unsupported  CHCS  ou^atient  visits  that  we  reviewed, 
did  not  satisfy  the  DoD  criteria  as  a  countable  visit  because  the  patient’s  medical 
records  did  not  contain  documentation  needed  to  verify  the  occurrence  of  one  or 
more  visits  to  an  authorized  outpatient  clinic.  The  other  100  errors  were  due  to 
a  variety  of  reasons  such  as  double  counting  of  visits  and  counting  invalid 
telephone  consultations. 


Table  1.  Outpatient  Visit  Comparison 

Military  Treatment  Facilities 

CHCS 

Outpatient 

Visits 

UnsuDDorted  Visits 

Pensacola  Naval  Hospital 

430 

272 

Walter  Reed  Army  Medical  Center 

244 

122 

Wilford  Hall  Air  Force  Medical  Center 

324 

212 

Totals 

998 

606 

In  addition,  CHCS  outpatient  visits  classified  as  telephone  consultations  did  not 
satisfy  the  DoD  countable  visit  criteria.  For  example,  we  reviewed  61  CHCS 
telephone  consultation  records  from  the  Wilford  Hall  Medical  Center  to 
determine  validity.  Of  die  61  telephone  consultation  records  reviewed,  36  were 
incorrectly  recorded  in  CHCS  as  countable  ouqiatient  workload  units.  Our 
analysis  of  the  36  invalid  telephone  consultations  identified  diat  the  outpatient 
worldoad  units  were  not  countable  for  some  of  the  following  reasons: 

•  telephone  consultations  involved  no  interaction  between  a  healthcare 
provider  and  a  patient,  such  as  messages  that  were  left  on  answering 
machines, 

•  telephone  consultations  were  counted  for  scheduling  patient’s 
appointments,  and 

•  telephone  consultations  were  counted  for  die  refill  of  prescriptions, 
rather  than  prescription  renewals,  which  are  countable  ouqiatient 
visits. 
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Level  of  management  controls.  The  level  of  management  controls  over  CHCS 
outpatient  visits  varied  by  DoD  military  tr^tment  facilities.  There  was  no 
consistent  approach  to  ensuring  the  reliability  of  the  ou^atient  data.  Of  the  five 
DoD  military  treatment  facilities  CHCS  host  sites  visited,  two  had  implemented 
limited  management  controls  to  ensure  the  reliability  of  the  CHCS  ou^atient 
visit  data.  The  Portsmouth  Naval  Medical  Center  and  the  Wilford  Hall  Air 
Force  Medical  Center  implemented  limited  management  controls.  Walter  Reed 
Army  Medical  Center,  Blanchfield  Army  Community  Hospital,  and  the 
Pensacola  Naval  Hospital  had  not  established  management  controls  to  ensure  the 
reliability  of  the  CHCS  outpatient  workload  data. 

DoD  military  treatment  facilities  that  implemented  management 
controls.  The  Portsmouth  Naval  Hospital,  Data  Quality  and  Analysis 
Department,  has  two  staff  members  that  visit  clinics  to  evaluate  daily  operations 
and  review  outpatient  visit  data.  The  Wilford  Hall  Medical  Center,  Resource 
Management  Office,  has  one  staff  member  that  conducts,  on  a  part-time  basis, 
limited  reviews  of  the  quality  of  CHCS  ouqiatient  visits.  Additionally,  the 
Wilford  Hall  Medical  Center,  Health  Records  Compliance  Element,  has  one 
staff  member  annually  review,  on  a  part-time  basis,  the  quality  of 
documentation  contained  in  the  outpatient  medical  records  to  comply  with  the 
Joint  Commission  on  Accreditation  of  Healthcare  Organizations  standards.  The 
Joint  Commission  on  Accreditation  of  Healthcare  Organizations  is  the  nation’s 
predominant  standards  setting  and  accrediting  body  in  healthcare  that  evaluates 
and  accredits  healthcare  entities  as  to  quality  of  care.  The  Joint  Commission  on 
Accreditation  of  Healthcare  Organizations  requires  adequate  documentation  for 
patient  medical  records.  Although  these  reviews  are  commendable,  they  are  not 
adequate  to  ensure  the  reliability  of  the  CHCS  outpatient  visit  data.  For 
example,  the  Resource  Management  Office  and  the  Health  Records  Compliance 
Element  staff  do  not  perform  reconciliations  between  CHCS  outpatient  visit  data 
and  the  supporting  documentation  contained  in  the  patient  medical  records.  We 
did  not  perform  tests  of  management  controls  at  the  Portsmouth  Naval  Hospital. 

Need  for  CHCS  Outpatient  Workload  Data  Quality  Assurance 
Program 

The  Office  of  the  Assistant  Secretary  of  Defense  (Health  Affairs)  and 
specifically  the  Project  Management  Office  for  the  CHCS  had  not  provided 
guidance  for  developing  and  implementing  a  CHCS  ouqiatient  visit  quality 
assurance  program.  A  data  quality  assurance  program  should  consist  of 
management’s  policies  and  procedures  to  provide  reasonable  assurance  that 
CHCS  outpatient  visit  data  are  being  reliably  recorded  and  reported.  In 
addition,  the  CHCS  outpatient  visit  quality  assurance  program  should  clearly 
delineate  the  specific  steps  that  management  designed  and  prescribed  to  provide 
reasonable  assurance  that  its  policies  and  procedures  will  be  implemented  and 
goals  achieved. 

An  effective  quality  assurance  program  over  the  reliability  of  the  CHCS 
outpatient  visits  data  will  provide  Ae  DoD  military  treatment  facilities 
reasonable  assurance  that  the  medical  records  contain  the  patient’s  complete 
medical  history  documentation.  In  addition,  a  complete  medical  history  will 
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provide  the  DoD  military  treatment  facilities  a  clear  and  acceptable  audit  trail 
between  the  CHCS  ou^atient  visit  data  and  the  patient’s  medical  records.  An 
adequate  audit  trail  will  also  improve  the  auditability  of  the  CHCS  outpatient 
workload  data  for  compliance  with  financial  statement  requirements. 

At  a  minimum,  effective  management  controls  to  ensure  the  reliability  of  the 
CHCS  outpatient  visit  data  should  include  ongoing  reviews  of: 

•  patient  medical  files  for  proper  documentation  and  reconciliation  with 
CHCS, 

•  end-of-day  workload  reports  for  accuracy,  and 

•  telephone  consultations  for  workload  validity. 

The  results  of  these  reviews  should  be  monitored  and  reported  to  higher  level 
management. 

Standardization  of  Data  Elements  for  Appointments 


The  DoP  military  treatment  facilities  management  controls  over  CHCS 
ouqiatient  visits  were  also  inadequate  because  Office  of  the  Assistant  Secretary 
of  Defense  (Health  Affairs)  did  not  develop  standardized  appointment  data 
elements  for  clinics  that  provide  comparable  medical  care.  The  individual 
clinics  within  the  DoD  military  treatment  facilities  assign  the  data  element  codes 
based  on  clinical  needs  and  preferences,  rather  than  standard  DoD  guidance. 
Appointment  type  is  an  example  of  a  data  element. 

We  compared  s^pointment  type  data  elanents  tiiat  the  Blanchfield  Army 
Community  Hospital,  Fort  Campbell,  Kentucky;  Pensacola  Naval  Hospital, 
Pensacola,  Florida;  and  Wilford  Hall  Air  Force  Medical  Center,  San  Antonio, 
Texas,  DoD  military  treatment  facilities  assigned  to  their  allergy,  audiology, 
cardiology,  dermatology,  ophthalmology,  and  optometry  clinics.  Our  analysis 
identified  that  appointment  type  data  elements  varied  widely  by  DoD  military 
treatment  facilities  and  by  clinics. 

There  was  little  consistency  and  standardization  of  appointment  types  used  ' 
between  the  three  DoD  military  treatment  facilities  and  their  6  clinics. 
Specifically,  there  were  a  total  of  541  appointment  types  used  between  the 
6  clinics.  However,  there  were  only  3  instances  where  the  t^pointment  types 
were  common  among  the  3  DoD  military  treatment  facilities  and  the  6  clinics 
(see  Table  2  for  the  details).  Some  of  the  diHerences  between  the  total  number 
of  appointment  types  are  attributable  to  the  level  of  medical  care  that  the  clinics 
provide  to  authorized  patients. 
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Table  2.  Appointment  Type  Variations 

Number  of  Appointment  Types  Number  of  Common 


Clinics 

Blanchfield 

Pensacola 

WUfordHall 

Total 

Allergy 

12 

26 

26 

64 

1 

Audiology 

21 

12 

48 

81 

0 

Cardiology 

9 

33 

97 

139 

0 

Dermatology 

16 

14 

38 

68 

1 

Ophthalmology 

40 

24 

46 

no 

0 

Optometry 

.24 

J9 

_26 

_79 

J 

Totals 

122 

138 

281 

541 

3 

Standard  Appointment  Data  Elements  Lnprove  Comparability.  The  DoD 
military  treatment  facilities  inability  to  establish  stand^d  appointment  data 
elements  creates  problems  for  managing,  controlling,  and  analyzing  CHCS 
outpatient  data,  DoD  officials  may  not  be  able  to  perform  cost-effectiveness 
studies  of  particular  medical  procedures  based  on  appointment  types  because  the 
appointment  type  codes  vary  between  instellations  for  similar  medical  care.  In 
addition,  the  inability  to  establish  standard  appointment  data  elements  may 
create  a  potential  reliabili^  problem  by  causing  the  CHCS  outpatient  data  to  be 
misstated.  For  example,  Wilford  Hall  Medical  Center,  Audiology  Clinic 
classified  ward  rounds  as  a  countable  outpatient  visits.  However,  DoD  guidance 
states  that  ward  rounds  are  not  countable  visits.  Hierefore,  standard 
appointment  data  elements  will  reduce  the  cases  where  the  DoD  military 
treatment  facilities  clinics  include  noncountable  outpatient  visits  such  as  ward 
rounds  in  CHCS  as  countable  outpatient  visits. 

Management  Efforts  to  Improve  CHCS  Outpatient  Data 

The  Office  of  the  Assistant  Secretary  of  Defense  (Health  Affairs)  is  initiating 
actions  to  correct  the  impediments  to  achieving  an  unqualified  audit  opinion 
identified  in  this  audit  report.  Hie  calculation  of  an  estimate  of  the  military 
retirement  healA  benefits  liability  is  part  of  the  DoD  effort  to  develop 
alternative  methods  for  achieving  an  unqualified  qudit  qiinion.  A  military 
retirement  health  benefits  working  group,  composed  of  representatives  from  the 
Under  Secretary  Defense  (Comptroller);  die  Assistant  Secretary  of  Defense 
(Health  Affairs);  the  Office  of  the  Actuary,  DoD;  (he  Inspector  General,  DoD; 
and  Ae  General  Accounting  Office;  have  held  a  series  of  meetings  to  address  the 
concerns  of  incomplete  and  unreliable  CHCS  outpatient  workload  data. 
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Health  Affairs  Officials  briefed  the  Military  Dq)artment  Surgeon  Generals  on 
the  audit  results  and  established  management  controls  over  outpatient  data  as  a 
high  priority.  Also,  Health  Affairs  officials  plan  to  develop  and  issue  a 
management  plan  for  correcting  data-related  management  controls  deficiencies. 

CHCS  Outpatient  Data  Impact  on  Financial  Statements 

DoD  management  is  making  progress  to  correct  data-related  management 
controls  deficiencies  that  are  impediments  to  DoD  a:Chieving  an  unqualified  audit 
opinion.  However,  the  CHCS  ouqiatient  data  does  not  provide  a  reliable 
database  for  developing  the  liabili^  estimate  and  if  used  to  develop  the  liability 
estimate  would  represent  an  impediment  to  our  review  and  oversight  of  the 
actuarial  liability  computation.  This  problem  constitutes  a  significant 
impediment  to  achieving  unqualified  audit  opinions  on  both  Ae  DoD  and 
Government-wide  financial  statements  for  IT  1999  and  beyond. 

Management  Comments  on  the  Finding  and  Audit  Response 

Assistant  Secretary  of  Defense  (Health  Affairs)  Conunents.  The  Assistant 
Secretapr  of  Defense  (Health  Affairs)  partially  concurred  with  the  overall 
conclusion  that  the  CHCS  data  was  unreliable  for  use  in  developing  the  Military 
Retirement  Health  Benefits  liability  estimate.  The  Assistant  Secretaiy  stated  that 
the  CHCS  ouq>atient  workload  data  could  be  validated  through  additional 
documentation  not  found  in  the  patient’s  medical  record.  The  Office  of  the 
Assistant  Secretary  of  Defense  (Health  Affairs)  and  TRICARE  Management 
Activity  has  tested  this  assumption  using  corroborative  evidence  that  tiiey  feel 
will  better  determine  the  reliability  of  CHCS  outpatient  data.  In  addition,  die 
Assistant  Secretary  believes  that  the  CHCS  ou4)atient  workload  data  used  was 
reasonably  accurate  and  complete  for  calculating  the  health  benefits  liability 
estimate. 

Air  Force  Comments.  The  Department  of  the  Air  Force  agreed  with  the 
material  management  control  weaknesses  identified  in  the  audit  report.  The  Air 
Force  stated  it  will  assist  the  Office  of  the  Assistant  Secretary  Defense  (Health 
Affairs)  in  the  development  and  implementation  or  reestablishment  of 
management  controls,  as  necessary,  to  improve  the  quality  of  the  Air  Force 
data. 

Audit  Response.  We  agree  that  failure  to  include  documentation  in  the  patient 
medical  record  is  a  serious  deficiency.  However,  we  do  not  agree  that  other 
corroborative  data  can  be  used  to  substantiate  the  workload  data.  Corroborative 
evidence  outside  the  patient  record  only  suggest  that  a  specific  visit  may  have 
occurred.  However,  it  is  not  simply  that  a  visit  occurred,  the  nature  of  the  visit 
must  meet  certain  specific  criteria  before  it  can  be  counted  as  a  valid  workload 
visit.  Both,  the  type  of  care  and  administrative  data  should  be  documented  in 
the  patient  medical  record.  We  believe  official  patient  medical  records  should 
be  the  source  of  verification  that  visits  meets  workload  criterion.  The  collection 
of  corroborative  evidence  outside  the  patient  medical  record  is  often  an 
impractical  exercise  that  diverts  the  attention  of  senior  officials  at  health  care 
facilities  to  provide  the  data  from  untested  sources.  The  exercise  to  obtain  such 
data  stresses  the  administrative  capacity  of  the  Military  Treatment  Facilities 
involved.  Such  stress  could  ultimately  bring  pressure  on  the  record  reviewers  to 
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accept  inferior  evidence,  such  as  entries  in  the  Ambulatory  Data  System,  as 
reasonable  support.  The  Ambulatory  Data  System,  recently  implemented,  is 
acknowledged  by  Health  Affairs  as  an  incomplete  dkita  source.  Further,  the 
Assistant  Secretary’s  assertions  do  not  address  the  fact  that  10  percent  of  the 
claimed  visits  were  totally  enoneous.  We  also  agree  with  management  that 
there  was  no  effective  quality  assurance  program  to  ensure  that  the  CHCS  data 
was  accurate.  Without  an  effective  quality  assurance  program,  over  the  CHCS 
data,  we  conclude  that  the  CHCS  data  is  unreliable. 

Recommendations  and  Management  Comments 


We  recommend  that  the  Assistant  Secretary  of  Defense  (Health  Affairs): 


1.  Develop  and  implement  a  data  quality  assurance  program  for 
recording  and  reporting  the  Composite  Health  Care  System  outyatient 
workload  data  and  related  documentation.  For  example,  the  quality 
assurance  program  should  include  review  of  Composite  Health  Care  System 
end-of-day  workload  reports  for  accuracy,  recondliation  of  the  workload 
data  with  appropriate  medical  file  documentation,  and  a  foUow-'up  process 
to  assess  corrective  actions. 


Assistant  Secretary  of  Drfense  (Health  Affairs)  Comments.  The  Assistant 
Secretary  of  Defense  (Health  Affairs)  concurred  and  will  be  developing 
increased  quality  assurance  and  internal  management  control  programs  for  all 
Military  Health  System  information  systems. 


Air  Force  Comments.  The  Air  Force  agreed  with  the  recommendation  and 
stated  that  the  new  Air  Force  Instruction  41-210,  Patient  Administration 
Functions,  contains  guidance  on  the  implementation  of  management  controls  to 
help  establish  a  consistent  approach  to  ensuring  the  reliability  of  ouqiatient  data. 


2.  Develop  and  implement  standardized  appointment  data 
elements  in  the  Composite  Health  Care  System  for  sin^ar  medical  care  in 
like  clinics. 

Assistant  Secretary  of  Defense  (Health  Affairs)  Comments.  The  Assistant 
Secretary  of  Defense  (Health  Affairs)  concurred  and  established  a  Tri-Service 
workgroup  to  develop  DoD  standardized  ^pointmait  types  that  all  clinics  in  the 
Military  Health  System  would  used. 

Air  Force  Comments.  The  Air  Force  agreed  tiiat  the  lack  of  standardized 
appointment  elements  in  the  Composite  Health  Care  System  is  a  serious  problem 
and  has  Air  Force  Medical  Service  representatives  working  with  the  TRICARE 
Management  Activity  on  various  committees  to  address  these  issues. 
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3.  Conduct  tests  to  determine  if  the  quality  controls  mid 
standardized  appointment  data  elements  have  improved  after 
implementation  of  recommendations  1  and  2. 

Assistant  Secretary  of  Defense  (Health  Affairs)  Comments.  The  Assistant 
Secretary  of  Defense  (Health  Affairs)  concurred,  stating  that  ongoing 
management  reviews  and  monitoring  of  quality  controls  will  be  incorporated  in 
the  design  of  the  overall  data  quality  assurance  program. 

Air  Force  Conunents.  The  Air  Force  agreed,  stating  that  the  new  Air  Force 
Instruction  41-210  addresses  a  data  quality  assurance  program  at  each  Air  Force 
military  treatment  facility. 
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Appendix  A.  Audit  Process 


Scope 

Work  Performed.  Our  audit  focused  on  die  review  of  the  underlying  data 
supporting  the  outpatient  visit  data  that  the  military  treatment  facilities  recorded 
in  the  CHCS.  DoD  has  104  DoD  military  treatment  ^ilities  designated  as 
CHCS  host  sites.  DoD  military  treatment  facilities  designated  as  a  CHCS  host 
site  may  have  CHCS  reporting  responsibilities  for  one  or  more  milimry 
treatment  facilities.  For  example,  the  Walter  Reed  Medical  Center  is 
responsible  for  reporting  the  CHCS  outpatient  visit  data  for  the  Dewitt  Army 
(immunity  Hospital,  Fort  Belvoir,  Virginia.  We  conducted  our  review  at  five 
DoD  military  treatment  facilities  CHCS  host  sites: 

•  Walter  Reed  Army  Medical  Center,t Washington,  D.C.; 

•  Blanchfield  Army  Community  Hospital,  Fort  Campell,  Kentucky; 

•  Pensacola  Naval  Ho^ital,  Pensacola,  Florida; 

•  Portsmouth  Naval  Medical  Center,  Portsmouth  Virginia,  and  the 

•  Wilford  Hall  Air  Force  Medical  Center,  San  Antonio,  Texas. 

For  FY  1997  oud)atient  workload,  103  of  the  104  DoD  military  treatment 
facilities  serving  as  CHCS  host  sites  reported  33  million  CHCS  visits  to  clinics. 
The  National  Naval  Medical  Center,  Bethesda,  Maryland,  visits  were  not 
available  in  time  to  be  included  in  this  report,  llie  five  DoD  military  treatment 
facilities  CHCS  host  sites  that  we  visited  reported  6.2  million  of  the  33  million 
outpatient  visits  (19  percent).  Therefore,  we  selected  the  five  DoD  military 
treatment  facilities  CHCS  host  site  locations  discussed  in  this  report  for  review 
because  we  wanted  audit  coverage  of  each  of  the  Military  Departments  and 
because  of  the  volume  of  outpatient  medical  care  that  these  DoD  military 
treatment  facilities  provided  to  eligible  beneficiaries. 

We  reviewed  the  procedures  for  recording  and  reporting  outpatient  workload 
data  in  the  CHCS.  Specifically,  we  selected  and  reviewed  patient’s  authorized 
records  of  medical  treatment  to  determine  whether  the  DoD  military  treatment 
facilities  recorded  countable  oufoatient  visits  in  accordance  with  DoD  criteria. 
We  conducted  this  portion  of  our  review  at  the  Walter  Reed  Army  Medical 
Center,  Pensacola  Naval  Hospital,  and  the  Wilford  Hall  Air  Force  Medical 
Center  CHCS  host  sites. 

During  FY  1998,  DoD  reported  a  $223  billion  unfunded  actuarial  estimate  for 
the  military  retirement  health  benefits  liability  on  the  DoD  and  the 
Government-wide  consolidated  financial  statements.  Approximately 
$172  billion  of  the  $223  billion  of  the  FY  1998  estimated  militaty  retirement 
health  benefits  liability  accrued  from  future  outpatient  and  inpatient  medical  care 
that  the  DoD  military  treatment  facilities  are  expected  to  provide  to  eligible 
Weficiaries. 
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Limitations  to  Audit  Scope.  We  limited  our  tests  of  management  controls  over 
the  CHCS  outpatient  visit  data  to  medical  records  that  the  Walter  Reed  Army 
Medical  Center,  Pensacola  Naval  Hospital,  and  Wilford  Hall  Air  Force  Medical 
Center  had  on-hand  at  the  time  of  our  review.  We  limited  our  review  because 
some  patient  medical  records  that  supported  outpatient  visits  data  were  not 
readily  available  for  review  because  they  were: 

«  located  at  other  DoD  military  treatment  facilities, 

•  in  the  custody  of  the  patient,  or  were 

•  unavailable  for  other  reasons. 

Methodology 


Use  of  Computer-^Brocessed  Data.  We  used  the  FY  1997  computer-processed 
data  that  the  military  treatment  facilities  used  to  record  and  report  clinical 
workload  data.  We  did  not  validate  the  reliability  of  the  CHCS  management 
system  because  we  limited  our  use  of  the  data  to  testing  management  controls 
and  to  obtaining  an  understanding  of  the  procedures  that  the  Office  of  the 
Actuary,  DoD,  used  to  calculate  the  liability.  However,  not  validating  the 
reliability  of  the  CHCS  system  did  not  affect  the  results  of  this  audit. 

DoD-wide  Corporate  Level  Govemmrat  Performance  and  Results  Act 
Goals.  In  response  to  the  Government  Performance  Results  Act,  tiie 
Department  of  Defense  has  established  6  DoD-wide  corporate  level  performance 
objectives  and  14  goals  for  meeting  tiiese  objectives.  This  report  pertains  to 
achievement  of  the  following  objectives  and  goals. 

Objective:  Fundamentally  reengineer  the  Department  and  achieve  a 

21st  century  infrastructure.  Goal:  Reduce  costs  while  maintaining 

required  military  capabilities  across  all  DoD  mission  areas.  (DoD-6) 

DoD  Functional  Area  Reform  Goals.  Most  major  DoD  functional  areas  have 
also  established  performance  iniprovement  reform  objectives  and  goals.  This 
rqiort  pertains  to  achievement  of  the  following  functional  area  objectives  and 
goals. 


Objective:  Strengthen  internal  controls.  Goal:  Improve  compliance 
with  the  Federal  Managers’  Financial  Integrity  Act.  (FM-5.3) 

General  Accounting  Office  Higb  Risk  Area.  The  General  Accounting  Office 
has  identified  several  high  risk  areas  in  the  DoD.  This  report  provides  coverage 
of  the  Defense  Financial  Management  high  risk  area. 

Audit  Type,  Dates,  and  Standards.  We  performed  this  financial-related  audit 
from  May  1998  through  December  1998  in  accordance  with  auditing  standards 
issued  by  the  Comptroller  General  of  the  United  States,  as  implemented  by  the 
Inspector  General,  DoD.  We  included  tests  of  management  controls  considered 
necessary. 
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Contacts  During  the  Audit.  We  visited  or  contacted  organizations  within 
DoD.  Further  details  are  available  on  request. 

Management  Control  Program 


DoD  Directive  5010.38,  "Management  Control  Program,  "August  26,  1996, 
requires  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  those  controls. 

Scope  of  Review  of  Management  Control  Programs.  We  reviewed  the 
management  control  programs  to  identi^  the  quality  control  programs  and 
control  techniques  to  ensure  that  outpatient  visit  data  were  accurately  recorded. 
In  addition,  we  reviewed  the  Under  Secretary  of  Defense  (Personnel  and 
Readiness)  FY  1997  Annual  Statement  of  Assurance  and  management’s 
self-evaluation  as  it  ^plied  to  controls  over  recording  and  reporting  outpatient 
clinical  workload  data. 

Adequacy  of  Management  Controls.  We  identified  material  management 
control  weaknesses,  as  defined  by  DoD  Directive  5010.38.  The  Office  of  the 
Assistant  Secretary  of  Defense  (Health  Affairs)  did  not  develop  and  implement 
management  controls  to  ensure  the  reliability  of  CHCS  ouqiatient  workload 
data.  The  Walter  Reed  Army  Medical  Center,  Wilford  Hall  Air  Force  Medical 
Center,  and  the  Pensacola  Naval  Hospital  management  controls  did  not  ensure 
that  adequate  documentation  supported  the  outpatient  clinical  workload  data 
recorded  in  the  CHCS.  The  recommendations  in  this  report,  if  implemented, 
will  improve  management  decisions  and  the  accuracy  and  completeness  of  the 
outpatient  clinical  workload  data  that  the  Office  of  the  Actuary,  DoD,  uses  to 
calculate  and  estimate  the  military  retirement  health  benefits  liability  reported  to 
DFAS  Indianapolis  Center  for  inclusion  in  the  Statement  of  Financial  Position 
for  future  years.  A  copy  of  this  report  will  be  provided  to  the  senior  officials 

'  responsible  for  management  controls  at  the  DoD  military  treatment  facilities. 

Adequacy  of  Management’s  Self-Evaluations.  Management’s  self-evaluations 
did  not  identify  the  specific  material  management  control  weaknesses  identified . 
by  the  audit  because  the  DoD  military  treatment  facilities  evaluations  did  not 
address  quality  controls  of  the  CHCS  ouqiatient  workload  data. 

Summary  of  Prior  Audit  Coverage 


During  the  past  5  years,  the  General  Accounting  Office  issued  two  audit  reports 
and  the  Inspector  General,  DoD,  issued  one  aumt  report  discussing  the  DoD 
Military  Retirement  Health  Benefits  Liability  and  CHCS  issues,  respectively. 


General  Accounting  Office 

The  General  Accounting  Office  issued  two  reports  during  FY  1994  tiirough 
FY  1996  related  to  the  CHCS. 

GAO/AlMD-96-39,  "Defense  Achieves  Worldwide  Deployment  of  Composite 
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Health  Care  System,”  April  5,  1996 

GAO/AIMD-94-61,  “Defense’s  Tools  and  Methodology  for  Managing  CHCS 
Performance  Need  Strengthening,”  July  5,  1994 


Inspector  General 

Inspector  General,  DoD,  Report  No.  99-010,  “DoD  Military  Retirement  Health 
Benefits  Liability  for  FY  1997,”  October  13,  1998. 
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Appendix  B.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Under  Secretary  of  Defense  (Personnel  and  Readiness) 

Assistant  Secretary  of  Defense  (Health  Affairs) 

Office  of  the  Actuary,  DoD 
Assistant  Secretary  of  Defense  (Public  Affairs) 

Director,  Defense  Logistics  Studies  Information  Exchange 

Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 

Surgeon  General,  Department  of  the  Army 

Auditor  General,  Department  of  the  Army 

Commander,  Walter  Reed  Army  Medical  Center 

Commander,  Blanchfield  Army  Community  Hospital 

Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 

Surgeon  General,  Department  of  the  Navy 

Auditor  General,  Department  of  the  Navy 

Commander,  Pensacola  Naval  Hospital 

Commander,  Portsmouth  Naval  Medical  Center 

Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Surgeon  General,  Department  of  the  Air  Force 
Auditor  General,  Department  of  the  Air  Force 
Commander,  Wilford  Hall  Medical  Craiter 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Finance  and  Accounting  Service 

Director,  Defense  Finance  and  Accounting  Service  Indian^olis  Center 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
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Non^Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 
General  Accounting  Office 

National  Security  and  International  Affairs  Division 
Technical  Information  Center 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 


Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Committee  on  Governmental  Affairs  >; 

House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Inform 

House  Subcommittee  on  Government  Management,  Information,  and  Technology, 
Committee  on  Government  Reform 

House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 


t 
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Assistant  Secretary  of  Defense  (Health 
Affairs)  Comments 


THE  ASSISTANT  SECRETARY  OP  DEFENSE 
WASHINOTON,  0.  C  tOSOMIM) 


health  ArFAIH* 


MEMORANDUM  FOR  DIRECrOR,  FINANCE  AND  ACCOUNTING  DIRECTORATE 
OFFICE  OF  THE  INSPECTOR  GENERAL,  DOD 

SUBJECT:  Audit  Report  on  Ddta  Supporting  the  FY 1998  DoD  Military  Retirement  Health 
Benefits  liability  Estimate  (Project  No.  $FA'>2016) 


This  memorandum  is  In  respmise  to  your  request  for  management  conunents  on  the 
subject  draft  audit  report.  This  office  generally  concurs  with  all  of  the  tecommmidadons  in  the 
report,  and  partially  concurs  with  the  overall  conclusion  that  the  quality  of  the  Compasite  Health 
Care  (CHCS)  system  outpatient  visit  data  was  unreliable  for  use  in  developing  the  military 
retirement  health  benefits  Uabllity  estimate.  We  believe  that  the  outpadeot  workload  data  from 
CHCS  can  be  validated  through  addidonal  documentation  not  found  in  the  padeDt*s  medical 
ncord,  and  the  current  InformaUon  is  reasonably  accurate  and  complete  for  calculating  the  health 
benefits  liability  estimate. 

The  audit  report  provides  a  valuable  tool  to  highlight  and  emphasize  the  need  for 
improving  data  quality  throughout  the  Military  Health  System  (MHS).  The  identification  of 
management  control  material  weaknesses  will  be  addressed  immediately  and  is  t  high  priority  for 
OASD  (Health  Affairs).  Wje  will  establish  and  implement  quality  assurance  programs  and 
internal  management  controls  necessary  to  Increase  the  reliiibiUty  of  the  CHCS  outpatient 
workload  and  other  data  in  the  MHS  automated  systems. 

We  undentand  the  importance  of  an  unqualified  audit  opinirm  on  the  Govemment-^wide 
financial  statements,  and  will  develop  and  implement  the  recommendations  in  the  draft  audit 
report  to  improve  the  quality  of  our  data.  We  will  cootinue  to  work  with  the  DoD  Office  of  the 
Actuary,  DoDlG,  Comptroller  and  GAO  to  review  and  develop  alternative  methodologies  Co 
calculate  the  retirement  liability  in  order  Co  obtain  an  unqualified  audit  opinion.  Thank  yoii  for 
the  opportunity  to  provide  conunents  on  this  draft  report.  Our  specific  comments  and 
reconunendations  are  in  the  attadhment.  The  point  of  contact  for  this  matter  is  Mr.  Ed  Chan.  He 
may  be  reached  by  phone  at  (703)  S8 1*1724  or  e-mailed  at  edmund.chan^tma.osd.mil. 


Attachment: 
As  Stated 


10  DRAFT  AUDIT  REPORT  -  DATED  December  23, 1998 
(PROJECT  NO.  8FA*2016) 

^aU  Supporting  the  FY 1998  Military  Retlremeitt  Health  Benefits  Liability  Estimate 

OASD  (HEALTH  AFFAIRS)  AND  TRICARE  MANAGEMENT  ACnVlT Y  (TMA) 
COMMENTS  TO  THE  AUDIT  RESULTS 

In  general,  Health  Affairs  (HA)  and  TRICARE  Management  Activity  (TMA)  concur  with  most 
of  the  findings  and  recommendations  in  the  dr^  audit  report.  However,  we  believe  that  the 
Composite  Health  Cart  system  outpatient  visit  data  is  reasonably  reliable  for  use  in  developing 
the  n^litary  retirement  health  beneHts  liability  estimate. 

The  review  of  the  CHCS  outpatient  visits  found  that  a  substantial  number  of  visits  were  not 
documented  in  the  patient's  outpatient  medical  record.  This  is  a  serious  finding  and  needs  to  be 
corrected  through  increased  oversight  and  development  of  management  controls  to  ensure 
compliance  with  established  policies.  However,  we  believe  there  is  considerable  and  sufficient 
documentation  in  other  than  the  outpatient  medical  record  that  provides  vididaUon  of  a  CHCS 
visit  Over  the  past  few  months,  we  tested  our  assumptions  at  three  medical  centers.  Most 
recently,  representatives  from  the  Office  of  the  Actuary,  DoDIO,  O AO,  and  Health  Afhurs  met 
with  staff  at  the  National  Naval  Medical  Center  (NNMC),  Bethesda,  Maryland.  We  obtained  the 
assistance  of  the  local  CHCS  host  administrator,  an  Ambulatory  Data  System  (ADS)  expert,  and 
several  health  care  providers. 

AU  of  the  visitors  to  NNMC  agreed  that  there  is  considerable  corroborative  daU  regarding  a 
patient  visit  even  when  a  medical  record  is  not  available.  We  are  currently  working  with  the 
DoDlG  and  GAO  to  design  a  patient-based  statlstiMl  sampling  plan  to  measure  the  reliability  and 
completeness  of  the  Military  Health  System*s  CHCS  outpatient  visits.  In  addition  to 
documentation  in  the  medical  outpatient  record,  the  DppIG  and  GAO  are  reviewing  the 
acceptability  of  other  information  from  electronic  and  clinic  maintained  documents.  The 
outcome  of  this  survey  would  better  detennine  the  leliabUity  of  the  CHCS  outpatient  data  and 
support  visit  validation.  We  also  plan  to  use  the  sampling  results  to  measure  the  impact  on  the 
estimate  of  the  outpatient  portion  of  the  retiree  health  liability  related  to  MTP  care. 


Management  Controls  Over  Military  Treatment  FadUtles  Outyatlent  Visit  Data 
Guidance  for  Counting  Outyatlent  Visits,  CONCUR 

The  current  visit  criteria  used  in  the  Medical  Expense  and  Performance  Reporting  System 
manual  was  developed  In  1995.  The  guidance  for  determining  countable  visits  is  being  revised 
to  reflect  changes  in  the  health  care  environment.  The  Military  Health  System  (MHS)  is  in  the 
process  of  transitioning  to  the  electronic  record  and  the  hard  copy  medical  record  will  be  obsolete 
within  a  few  years.  The  revised  visit  definition  will  need  to  more  clearly  define  adequate 
documentation  to  include  the  acceptance  of  electronic  or  computerized  documentation  and 
indicate  where  additional  validating  documentation  can  be  found. 
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QuaUty  of  CHCS  OutpaiScnt  Workloaa  Date  if  Unreliable.  PARtlALLY  CONCUR 

We  agree  that  the  MHS  needs  to  develop  ai^  Increase  management  controls  to  improve  the 
accuracy  and  quality  of  the  infomadon  from  all  of  onr  Infonnadon  systems.  However,  for  the 
purpose  of  calculating  the  liability  estimate,  unless  CHCS  visit  miscalculations  are 
disproportionately  spread  across  active  duty  or  retirees,  the  overall  calculation  will  be  reasonably 
accurate.  The  DoD  Actuary  Office  and  TMA  will  be  conducting  a  randomized  survey  of  the  FY 
98  CHCS  outpatient  visits  to  assess  the  quality  of  the  data  and  develop  a  dau  sensitivity  analysis 
to  adjust  the  estimate  if  necessary. 

Most  of  the  unsupported  CHCS  outpatient  visits  that  were  reviewed  did  not  satisfy  the  DoD 
criteria  as  a  countable  visit  because  the  patient*s  medical  record  did  not  contain  the  proper 
documentation.  However,  other  sources  for  documeiiiation  that  the  visits  occurred  were  not 
investigated.  Tbdse  sources  include  convenienoe  files  that  the  physicians  maintain, 
documentation  in  the  file  rooms,  and  other  electronic  documentation  auch  as  Ambularory  Data 
System  (ADS)  records.  The  inclusion  of  these  other  documents  will  substantially  incie&«;e  the 
validation  of  the  visit  counts. . 

Other  errors  such  as  double  oouniing  of  visits  and  counting  invalid  telephone  consultations 
reflect  a  need  to  ensure  appropriate  education  and  training.  Clear  direction  and  standard 
procedures  for  making  appointments  and  counting  telephone  consults  will  need  to  be 
communicated  to  alt  individuals  using  the  CHCS  system. 


Need  for  CHCS  Outpatient  Woridoad  Date  Quality  Assurance  Program.  CONCUR 

Development  of  consistent  management  controls  and  quality  assurance  program  to  ensure  the 
reliability  of  data  from  MHS  automated  systems  is  a  inq|or  priority  in  the  MHS.  The  Office  of 
Health  Affairs  and  the  TRICARE  Muiagement  Activity  have  begun  ti^  process  to  establish  DoD 
wide  management  control  and  quality  assurance  programs.  A  Data  Quality  Integrated  Program 
Team  composed  of  HA,  TMA  and  Service  representatives  has  been  established  that  will  be 
responsible  for  developing  quality  assurance  programs  for  the  MHS  automation  systems.  A 
project  master  plan  is  being  developed  with  milestones  and  schedules  for  all  key  stakeholders. 
Routine  reporting  on  the  status  of  the  phases  and  the  milestones  to  the  Milestone  Decision 
Authority  (MDA)  and  the  Deputy  Surgeons  General  will  be  required. 

Stendardization  of  Date  Elemantt  for  Appointments.  CONCUR 

\ 

In  the  past,  MTB  have  been  given  the  latitude  to  develop  their  appointment  systems  to  suit  their 
needs  and  individual  situations  at  the  local  level  Over  tiie  years  tiie  individual  creativity  of  each 
Service  and  each  MTF  within  ihe  Services  led  to  the  current  situation  where  we  have  an 
excessive  number  of  appointment  types  throughout  the  MHS  with  many  different  names  to 
describe  the  same  encounter. 

The  number  of  appointment  types  needs  to  be  reduced  to  sin^Ufy  the  ^>pointing  process  and 
allow  for  better  comparisons  of  facilities  and  performance  measurement.  Standardization  is  a 


complex  project  that  requites  a  uniform  set  of  business  rules  and  other  stnictural/jpolicy  changes 
before  it  can  be  accomplished.  It  will  also  require  high  level  support,  resources,  and  a  lot  of  site 
preparation  (data  clean  up)  to  be  implemented  properly. 

Standardized  Appointment  Data  Elements  Improve  Comparability.  CONCUR 

Standardized  appointment  types  will  help  to  ensure  a  uniform  approach  to  workload 
accountability  b^ause  the  critical  data  elements  within  the  clinic  and  iq)pointinent  type  profiles, 
e.g.,  Woikload  Type:  (count  vs.  non'Count)  can  be  set  the  same  for  all  Slices.  This  will 
facilitate  consistency  between  the  Services  and  MTB  when  workload  counts  are  necessary  for 
cost  calculations  such  as  the  retiiement  liability  calculation. 

Develop  and  Implement  standardized  appointment  data  elements.  CONCUR 

In  January  199P,  work  began  to  establish  standardization  of  appointment  types  to  fit  current 
business  practices  throughout  DoD.  The  Tri-Service  workgroup  Is  In  the  process  of  developing  a 
plan,  using  an  Integrated  Product  Team  iq;»proach,  to  implement  the  standardized  appointment 
types  and  data  elements  bc^nning  within  Region  1.  An  implementation  plan  will  be  developed 
by  summer  1999.  The  goal  is  to  implement  the  standardize  appointment  types,  data  elements, 
and  clinic  names  across  the  MHS. 

RECOMMENDATIONS: 


1.  Develop  and  implement  a  data  quality  assurance  prograin  for  recording  and  reporting  the 
Composite  Health  Care  system  outpatient  workload  data  and  related  documentation. 

Concur.  The  Office  of  the  Assistant  Secretary  of  Defense  (Health  Affairs)  and  the 
TRIG  ARE  Management  Activity  are  developing  increased  quality  assurance  and  internal 
management  control  programs  for  all  MHS  information  systems. 

2.  Develop  and  implement  standardized  appointment  data  elements  in  the  Composite  Health 
Caro  system  for  similar  medical  care  in  like  clinics. 

Concur.  A  Tri-Service  workgroup  has  been  established  to  develop  DoD  standardized 
appointment  types  to  be  used  by  all  clinics  in  the  Military  Health  System. 

3.  Conduct  tests  to  determine  if  the  quality  controls  and  standardized  appointment  data  elements 
have  improved  after  implementation  of  recommendations  1  and  2. 

Concur*  Ongoing  management  review  and  monitoring  of  quality  controls  wilt  be 
incorporated  in  the  design  of  the  data  overall  quality  assurance  program. 
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Material  Control  Wealuictf  Conunenta 


The  draft  audit  i^it  identified  material  management  control  wealqiesses.  First,  it  states  that  the 
Office  of  the  Assistant  Secietaty  of  Defense  (Health  Affairs)  did  not  develop  and  implement 
management  controls  to  ensure  the  reliability  of  CHCS  outpatient  workload  data. 

Concur.  The  TRICARE  Mi^iagement  Activity  will  oversee  the  estabUshmeot  and 
implementation  of  a  comprehensive  strategy  for  manageinent  controlf  that  provides  reasonable 
assurance  that  the  iCHCS  outpatient  workload  is  reliable.  This  will  include  a  system  of  guidance, 
instructions,  procedures,  and  appropriate  management  oversight  to  ensure  that  the  management 
controls  are  operational. 

The  Walter  Reed  Army  Mescal  Center.  Wttford  HaU  Air  Fmce  Medical  Center,  and  the 
Pensacola  Naval  Hospital  mana^ment  controls  did  not  ensure  that  adequate  documentation 
supported  the  outpatient  clinical  workload  recorded  in  CHCS. 

Concur.  The  quidity  assurance  program  and  manageinent  controls  will  be  implemented 
throughout  the  MHS  and  a  standardized  system  will  be  available  to  ensure  the  reliability  of  the 
outpatient  clinical  woricload  In 
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Department  of  the  Air  Force  Comments 


MBl^0RA^^)yMroBASSmva«INSI«ClOR  GENERAL  FOR  AUCar  «30 

PRCM:  HQUSAWSO 

110Liik«Av«nw,  Room  400 

BolUivAFB.DC  SQOn-TOn 

SUBIECT:  AudilReportoo  dtoFY  19M  DoDMUkny RecironWMt 

Health  Benefiia  LMbilUy  Eltioule  Object  No.  8FA-201^,  (dated  2S  Dee  98) 

We  have  leviewed  the  IntitectM’  Oeiwial  dG)  Audit  RepoR  (Px^ect  No.  8PA-20ld)  and  the 
following  oommentu  «n>  provided  to  lecotnineiidiHooi; 

EtecomowoMlon  l:  I>vefopmd  io^kaneAto  data  quaUtyHsoianeopeograni  for leoordiog 
and  npoRing  tike  CksfiOtifo  Heahh  Cm  Syiioin  (GHCS)  ouvodeot  worUoad  data  «imI  lelaied 
dcpowneidatlon.  For  eauiffo.  (be  <)wQQf  awnrance  pipgam  akooU  ioehide  ra view  of  CHCS 
end^f-day  woiicioad  repons  for  accun^,  leoonoiliiitlon  of  the  woihload  dal  a  with  appropriate 
!  medical  file  doemnenlatioo,  and  afoUoW'Upprocctt to  assestcoriectlveaciloiK. 

t 

Rc^jiso:  Concm*.  Tfae  new  41-2  iO;  Patient  Adinldstivt  ton  I^nCtkHis.«}nt&inft 

guidance  on  the  in^kmeAtMton  of  management  conifois  (o  eitabJiah  «  constitent  «^p|iiOQOli 

10  ensuring  the  reJtobUlty  ofootpettonldUU.  Specifiqdly,  pc«4act  line  imitagen  (PLM)  will  bo 
i^ointcd  nt  eedi  MAJOCMM  aud  mcdinl  Irotameoc  facility  (MTP)  tob&hdd  M^nilbk  end 
accountable  I^IhudAtft.  Inaddltton,  l«ml audit  prooedumwU be reeaoOjUi^ 
to  validate  tlie  occurrence  of  a  recorded  CIO  outpatient  vtoiL  The  levhed  API 
expected  lo  be  pubUfiHed  cbift  aimmer.  BCD;  July  1999. 

Recommendation  2i  Devotop  and  implejnent  atandarifiacd  appolntiueot  data  etoments  in 
CHC5  for  sirrular  tragical  core  iu  like  cfiuice. 

Reeponae:  Concar.  Tho  Jack  pf  standardiaad  appolntxncnt  data  PkmeAts  In  CHCS  to  a 
miott&concera;  however tlito  tosuo  to  beyoed  ourdliect  contxok  Wc  folly  luppcct  foeeffons  by 
Che  TRICARE  MiuMgeineiit  Activity  (TMA)  to  ataudtodtoe  data  dtontemn  acroiB  the  Milknry 
Health  Services  System  (MHSS),  We  have  Air  Force  Medicd  Service  (APM9)  reprdentativer 
woddeg  with  TMA  on  vartoufi  comniittoca  Ip  address  theie  Issues. 

Rnoommendatton  3:  GbnduCt  tests  U>  MenulM  if  the  quality  conlroU  and  staudardlxed 
appointment  data  eleraems  inve  ixnpiovod  After  iii]a>Itinencaiilon  of  xecommendations  1  and  2. 

ReitponRe:  Ctoncur.  The  nOv/ An  41-210  will  address  ndMaqunlkyASSUtanceptcam 
imph^nimiacioA  at  each  Air  f^rce  MIF  and  it  will  leinforpe  the  Hnpprtaitce  and  requiicxnunt  to 
j  audk  outpatient  visit  date  to  ensure  reliability.  BCD:  July  199^. 
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of Maiu^ement Controls.  CtmcurmlhlboowerinlroMOftam 
idontHiedintheawfitO^tt.  control 

dcdalDw  wd ihi> acoufflcy »od coeylewntos of ouiya^clfaM iiwridoad dnJUb  Wowillid$ist 
OA^(HA)int?}&dovolopo)tc«KW  ImplemeodiioQ  orrocslabiU^hinontofiiUnt^^e^ 
thiooi^iovittheMUSS^  as  neoessaiy^io  taTprovciho^natiOf  of()^ 

1  i^budlho  EG  olToits  (oidoudfy  tfiesadaia  quallQr  dtscropaadcs  ttroiii^  diolr  tuidil 
(loccdnics.  MyPOCfi)rttve$ei»wa6!|  Col|^htdJ.Hqwitt^^ 


MICHA1SLK.WTO 

ll4orOcD0ril.t^ 

De^Su^pnOio 
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Audit  Team  Members 

The  Finance  and  Accounting  Directorate,  Office  of  the  Assistant  Inspector 
General  for  Auditing,  DoD,  produced  this  report. 
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